Updating Crypto to use WebCrypto API and to replace RSA with ECC#446
Merged
tegefaulkes merged 68 commits intostagingfrom Dec 8, 2022
Merged
Updating Crypto to use WebCrypto API and to replace RSA with ECC#446tegefaulkes merged 68 commits intostagingfrom
tegefaulkes merged 68 commits intostagingfrom
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR focuses on updating the crypto utilities used by PK. We've been hitting problems using RSA and node-forge utilities, and we should start using the standardised WebCrypto API. This won't fully solve cross platform cryptography because that will need to wait until we hit mobile platforms and deal with it by using WASM or other utilities.
There are some new features coming into this PR:
NodeIdis now finally the public key. This means you no longer have to acquire the public key separately from theNodeId. Once you have theNodeIdyou can use it for public key verification, and for encryption.@peculiar/webcryptois being monkey patched toglobalThis.crypto. This ensures that every library is using the same webcrypto backend and this includes CSPRNG and encryption/decryption facilities.Massive performance improvements in all areas:
Details
Old performance (node-forge):
New performance (web crypto):
Newer performance (libsodium):
Issues Fixed
--root-key-file#433tokensdomain and specialise tokens for Sigchain, Notifications, Identities and Sessions #481KeyRing,CertificateManager#472NodeIDhas changed. #386Tasks
ED25519as the root key@scure/bip39for generating recovery code and deterministically generating the ED25519 root key* Note that
panva/josedoes not allow randomness to be standardised, this means the JOSE library will need to be replaced in the future* For now all libraries will mostly end up using node's native randomness generation because we are running in node runtime
* The
panva/joselibrary can be replaced... it's just mostly implementing JOSE RFCs that's the issue, but we are only using a limited set, otherwise can fork the library to provide an alternative implementation. Alternatively we would need to monkey patch a global webcrypto runtime@peculiar/x509CertManagerand plug this into the TLS configuration.KeyRingclass which extracts all root key pair and KEM mechanism out ofKeyManager.KeyRingby extracting out tests fromKeyManager.test.ts.CertificateManagerto extract out root certificate functionality out ofKeyManager. It must take theKeyRingandDBas dependencies.CertificateManagerby extracting out tests fromKeyManager.test.ts.KeyManagerwithKeyRingif they only require theNodeId.CommandStartandCommandBootstrapneed's it's configs updated.ObservableofKeyRing, continue using theEventBusfor theKeyRing.* This requires changing
KeyManagerChangeDatatoCertificateManagerChangeDatafor now, as that's where the origin of renewing identity will come from.KeyManagerfor now, and plan a new issue for a newKeyManagerintended for secure computation usage and the management of arbitrary subkeys.[ ] 22. Sigchain needs to use the- Sigchain is being refactored, see Replace JOSE with our ownCryptoKeyby usingkeysUtils.importKeytokensdomain and specialise tokens for Sigchain, Notifications, Identities and Sessions #481src/claims/utils.ts:36createClaimtakes the private key as the PEM format, this needs to be updated to take a private key directly.CertManagermay have an expired current certificate occurring due not starting theCertManagerfor a while. This means we need to immediately renew the certificate uponstart. Right now this is not guaranteed. Need to add in some renewal logic that occurs automatically if the current certificate is now expired.pk agent statuscommandwrapWithPasswordwhen outputting the private key to show the key pairpk keys root, these should be showing the JWK for public key, and JWE for the private key, use dictionary formatting as well during human format, and JSON otherwisepk keys private,pk keys public,pk keys keypair. Theprivateandkeypaircommands should be taking a password for wrapping. This should take--password-pathor take from input prompt. The--format jsonshould produce a useful JSON dictionary. Forkeypairit should be{ publicKey: JWK, privateKey: JWKEncrypted }.pk agent statusproduces a recursive dictionary output for public key JWK.pk agent startandpk agent bootstrapcan use all new key ring configuration and cert manager configuration.randomSourceinto whereverIdSortableandIdRandomis being constructed. This ensures thatjs-idis usingkeys/utils/random.tsinstead of its own provided randomness.NodeId.src/tokensdomain replacing JOSE JWSsrc/tokenssrc/claimsdomain specialisingsrc/tokenssrc/claimstokensusing parsing functionsclaimsusing parsing functionsSigchainto use the new claims and tokensSigchainwith the new claims and tokens structureSigchainfor faster access for link identity and link nodeidentitiesto use the newtokensandclaimsIdentityInfoandNodeInfointogestaltsgestaltsto record indexed information acquired from discovery - Gestalt Link Schema Refactoring - Derived from JOSE replacement #492discoveryto use thetokensandclaims, in particular claim links and verifying claim links - Discovery Refactoring - Derived from JOSE replacement #493notificationsto use the newtokensGeneral.json,VaultShare.json, andGestaltInvite.jsonto useparse/generateutilities instead of JSON schema. These utilities should go into thenotifications/utils.ts. It can reference thevalidation/errors.ts.tokensdomain, and they should be "signed tokens"sessionsto use the newtokenscreateSessionTokenandverifySessionTokenwith calls to thetokensdomain. The session token should then be aSignedToken. The signature is being signed by a symmetric key. Not the private key.src/keys/utils/hashing.ts. (Replace node forge uses with these). Note that multiformats hashing may require a "webcrypto" polyfill, but we don't know for suregestaltGraph.setNode()for the agent's own node at the startup of the agent. - Updating Crypto to use WebCrypto API and to replace RSA with ECC #446 (comment)Sigchain.getClaimsandSigchain.getSignedClaimspagination testing problems: Sigchain Class API should provide paginated ordered claims by returning Array-POJO and indexed access #327 (comment)Testing
Tests must start using fast check arbitraries and where suitable model based testing:
Minimal tests for networking, grpc, nodes because we are likely to change it quite a bit in our next major rework of the networking with QUIC and RPC with JSONRPC.
Final checklist